Value in Health
○ Elsevier BV
All preprints, ranked by how well they match Value in Health's content profile, based on 11 papers previously published here. The average preprint has a 0.01% match score for this journal, so anything above that is already an above-average fit. Older preprints may already have been published elsewhere.
Gilson, F.; Osstyn, S.; Handels, R.
Show abstract
BACKGROUNDTransparency and credibility of health-economic simulation models is essential to inform reimbursement decisions. Model replication can support model transparency and credibility. Artificial intelligence (AI), particularly large language models, offers new opportunities to accelerate model replication. This led to the research question: "To what extent can the results of existing health-economic Markov models be replicated by models developed using generative AI for eliciting input parameters and code generation?" METHODSReplication was performed in three steps. First, a chain-of-thought prompting strategy in ChatGPT-4 was developed to replicate in R an open-source model co-developed by one of the authors and with publicly available code. Second, it was applied to replicate a model co-developed by one of the authors but without publicly available code. Third, it was applied to a model without the involvement of the authors and without publicly available code. A mixed- methods approach was employed in terms of qualitatively addressing the face validity of the prompt development and refinement and quantitatively assessing deviations between AI- generated and original model predictions. RESULTSThe first model required approximately one month to replicate, while adaptations to the second and third models took approximately two weeks each. Across the three models and 45 replications (15 per model), the average absolute relative deviations between ChatGPT-4 generated model predictions and published results were: [≤]14% for quality-adjusted life years and costs in the first model, [≤]7% in the second model, and [≤]28% in the third model. CONCLUSIONSOur approach could support more time-efficient model replication for reimbursement decision-makers, researchers or pharmaceutical companies. This could contribute to transparency and credibility of health-economic models.
Sharp, S.; Lokuge, K.; Elvidge, J.; Hudson, T.; Dawoud, D.
Show abstract
ObjectivesGenerative AI (GenAI) has emerged in the current decade as a paradigm-shifting technology with potential to transform the process of health economic evaluation (HEE), a resource-intensive element of health technology assessment. This systematic literature review aims to identify the current applications of GenAI in HEE and its potential advantages, challenges and limitations. MethodsWe searched Medline, Embase, EconLit, Cochrane Library, International HTA database and Epistemonikos for English-language, publicly available literature without date restrictions, and hand-searched the ISPOR presentations database (2023 to 2025) for articles describing or investigating the use of GenAI in HEE. Quantitative data on performance outcomes were collected along with qualitative data on stakeholder opinions and experience. ResultsWe identified 25 eligible studies: 18 primary studies, 6 narrative reviews and 1 expert opinion piece. The primary studies comprised 16 case studies and 2 qualitative studies. Over 90% of studies were conference abstracts published in 2024 from commercial authors. The emphasis across studies was on early exploratory research, particularly model replication. Where reported, execution time (3 studies), accuracy and error rate (7 studies), and user experience (4 studies) showed promising results across multiple use cases but there is a high risk of bias inherent in relying on conference abstracts with limited reporting, which warrants cautious interpretation. ConclusionThe current evidence landscape has revealed potential benefits of GenAI across multiple applications to health economics, but only sparse dissemination of early case study findings via conference submissions. Further research is needed to validate all use cases and to address perceived barriers to implementation.
Blissett, R. S.; Sullivan, W.; Subban, I.; Igloi-Nagy, A.
Show abstract
Cohort-level models in Microsoft Excel(R) remain the standard for cost-effectiveness modelling to inform health technology assessment (HTA), despite calls and rationale for more flexible approaches. Their limited ability to capture patient-level characteristics can, in the presence of patient heterogeneity or the need to track patient characteristics to accurately capture a technologys implications, introduce bias. Their continued prevalence is explained by key stakeholders familiarity with spreadsheet software, and the lower computational burden of cohort-level versus patient-level models. However, contemporary Excel functions have opened up possibilities for efficient calculations within native Excel that enable more flexible, patient-level approaches to be implemented in familiar spreadsheet-based software. Therefore, this tutorial aims to provide step-by-step guidance on how to implement a previously published and freely available individual-level discrete event simulation (DES) in Excel, using contemporary Excel functions and without any Visual Basic for Applications (VBA) code. Key Points for Decision-MakersO_LIPerceived and real requirements for cost-effectiveness models for HTA to be built in Excel may have led to overuse of cohort-level approaches, with probable bias implications for HTA decision-making. C_LIO_LIContemporary Excel functions now allow the efficient implementation and execution of patient-level model calculations within native Excel, without any VBA code. Such capabilities may reduce technical barriers across key stakeholders, enhance transparency, and ultimately lead to improvements in HTA decision-making. C_LIO_LIThis tutorial demonstrates provides step-by-step guidance on how to implement an efficient patient-level cost-effectiveness model in Excel without any VBA, with an executable model example included as supplementary material. C_LI
Lee, J. T.; Shen, T. K.-B.; Liu, V. T.-N.; Chen, T. H.-H.; Lee, C.-C.; Lu, C.; Huang, C.-W.; Atun, R.
Show abstract
Economic evaluations of artificial intelligence (AI) in healthcare are expanding rapidly, yet underlying costing methods remains heterogenous, and frequently incomplete for health technology assessment (HTA) and policy decision-making. In our systematic review of 55 studies published between 2010 and 2025, we found that fewer than half of the studies reported explicit costing methods; most pricing analyses failed to describe the basis of fees, subscription terms, or duration of coverage; and few analyses distinguished between average and incremental costs or accounted for economies of scale. Lifecycle expenditures--including development, validation, integration, maintenance, retraining, and decommissioning--were largely omitted, while electricity consumption, data hosting, and cloud infrastructure costs were almost never considered. Sensitivity analysis was the exception rather than the norm, and reporting of cost offsets such as reduced hospital admissions or workforce time savings was inconsistent. To address these gaps, we propose a 20-item reporting checklist to standardise the costing and pricing of AI interventions. The checklist complements existing HTA frameworks while capturing features unique to AI, such as continuous retraining, reliance on data infrastructure, and recurrent maintenance. We also introduce an AI Costing Inventory and Calculator that operationalises a lifecycle approach, enabling systematic recording of resource use, unit costs, inflation adjustments, and total and incremental costs, including offsets. These tools extend the emerging CHEERS-AI reporting framework by embedding a lifecycle perspective into costing, thereby enabling consistent estimation of resource and cos components and strengthening the methodological foundations of AI economic evaluation for policy use.
Morris, K.; Pennington, M.; Whitelegg, C.; Parkes, L.; Mahon, R.
Show abstract
BackgroundAccess to new oncology medicines is subject to delays in many countries due to lengthy appraisal processes. This study examines the cost to the Irish healthcare system of implementing a cost- sharing agreement to expedite access to oncology medicines. MethodsThe hypothetical cost of implementing an early access scheme in Ireland was estimated for oncology medicines commencing appraisal in 2022. The scheme would have required the manufacturer to cover the cost for the first 180-day period and provide a further rebate if costs per patient over the whole duration of access exceeded those that would have arisen from the finally agreed price. Costs of the new medicine and savings arising from any therapies displaced were estimated on a daily basis using data published in the technical summaries of the National Centre for Pharmacoeconomics, Ireland (NCPE) assessment for each medicine. ResultsThe scheme would have reduced the time patients waited to access oncology medicines by more than two years on average. Assuming new medicines attract a discount of 30% at reimbursement and that medicines with an existing agreement are subject to a 30% discount, and that a further discount of 10% would be negotiated at reimbursement for the new indication, the costs to the government over the duration of the scheme would have been {euro}61.9m. Earlier access would have generated an additional 1,621 quality-adjusted life years (QALY) over the lifetime of patients accessing the scheme, after discounting. Costs were sensitive to assumptions on discounts negotiated at reimbursement. Costs fell substantially if patients with private insurance were assumed to access care through that insurance. ConclusionProtracted assessment times lead to substantial health losses to patients with cancer in Ireland. A cost-sharing scheme would accelerate access to new treatments by more than two years and at costs which are unlikely to exceed {euro}62m.
Bowen, H. P.; O'Loughlin, G.; Schleicher, C.; Schulthess, D.
Show abstract
Background: The impact of the Inflation Reduction Act (IRA) upon late-stage developments has been assumed to be limited. The Congressional Budget Office's IRA analysis excluded post-approval innovation, potentially overlooking substantial economic risks to drug developers and declines in the availability of treatments in areas of high unmet medical need such as oncology. Methods: A total of 1148 secondary trials from 364 FDA-approved medicines, published from 2018 to 2025, were obtained from Biomedtracker and clinicaltrials.gov. Using fractional multinomial logit, we model the share distribution of secondary indication studies across 19 disease groups and assess the change in this distribution post-IRA. We also assessed the number of secondary treatment studies pre- vs. post-IRA using multiple linear regression. Results: After the IRA's introduction, small molecule follow-on studies in oncology exhibited a statistically significant 35% decline (R2 = .48, p < 0.014) and lead indication small molecule oncology approvals exhibited a statistically significant 27% decline (R2 = .70, p < 0.002). We also find a statistically significant 14% decline in the share of orphan oncology studies pre- vs. post-IRA (p<0.001). Research Conclusions: This study's results refute claims that the IRA would have minimal negative effects on patient access or late-stage biopharmaceutical R&D. We hope this study reinvigorates debate about the law's unintended consequences and encourages thoughtful policy solutions, as the IRA manifestly creates disincentives that negatively impact patients seeking needed new medicines, particularly those requiring cures addressing metastatic late-stage cancers.
Sen, S. E.
Show abstract
BackgroundDisease-modifying therapies (DMTs) for Alzheimers disease, including lecanemab and donanemab, have received regulatory approval in multiple jurisdictions. These therapies require complex diagnostic workup and safety monitoring, raising significant budget impact concerns for healthcare payers. No budget impact analysis specific to Ireland or comparable small European healthcare systems has been published. ObjectiveTo estimate the 5-year budget impact of introducing DMTs for early-stage Alzheimers disease in Ireland from the Health Service Executive (HSE) payer perspective. MethodsA budget impact model was developed following International Society for Pharmacoeconomics and Outcomes Research (ISPOR) guidelines. The model incorporated Irish epidemiological data, published drug prices, and healthcare resource utilisation estimates. Three treatment uptake scenarios (conservative, base case, optimistic) were modelled over a 5-year time horizon. Sensitivity analyses examined parameter uncertainty. ResultsFrom an estimated eligible population of 11,568 individuals with early-stage Alzheimers disease, annual budget impact in Year 5 ranged from {euro}12.8 million (conservative: 3% uptake) to {euro}89.6 million (optimistic: 20% uptake), with a base case estimate of {euro}35.8 million (8% uptake). Cumulative 5-year budget impact ranged from {euro}32.0 million to {euro}224.0 million. Drug acquisition costs represented 61% of total expenditure, with diagnostic and monitoring costs comprising 24% and 15%, respectively. Sensitivity analysis identified drug price, eligible population size, and treatment uptake as the most influential parameters. ConclusionsIntroduction of DMTs for Alzheimers disease will have a substantial but manageable budget impact on the Irish healthcare system, contingent on treatment uptake rates constrained by diagnostic capacity. Strategic investment in diagnostic infrastructure, phased implementation, and negotiated drug pricing could mitigate budgetary pressures while enabling patient access to these novel therapies. Key Points for Decision MakersO_LIThis is the first budget impact analysis of disease-modifying therapies (DMTs) for Alzheimers disease specific to Ireland, a small European healthcare system with constrained diagnostic capacity. C_LIO_LIAnnual budget impact ranges from {euro}12.8 million (conservative scenario) to {euro}89.6 million (optimistic scenario) in Year 5, representing 0.05% to 0.33% of the total Health Service Executive budget. C_LIO_LIDrug acquisition costs account for 58-65% of total expenditure, with diagnostic workup and safety monitoring comprising substantial ancillary costs. C_LIO_LIPhased implementation aligned with diagnostic infrastructure expansion could enable budget-neutral introduction through efficiency gains in dementia care pathways. C_LI
Gillett, P.; Mahar, R. K.; Tran, N. R.; Rosenthal, M.; IJzerman, M.
Show abstract
BackgroundDemonstrating safety and efficacy of new medical treatments requires clinical trials. But clinical trials are costly and may not provide value proportionate to their costs. In health systems with limited resources, it is important to identify the trials with the highest value. Tools exist to assess elements of a clinical trial such as statistical validity but are not wholistic in their valuation of a clinical trial. This study aims to develop a measure of clinical trials value and provide an online tool for clinical trial prioritisation. MethodsA search of the academic and grey literature and expert consultation was undertaken to identify a set of metrics to aid clinical trial valuation using multi-criteria decision analysis. Swing weighting and ranking exercises were used to calculate appropriate weights of each of the included metrics and to estimate the partial-value function for each underlying metric. The set of metrics and their respective weights were applied to the results of six different clinical trials to calculate their value. ResultsSeven metrics were identified: unmet need, size of target population, eligible participants can access the trial, patient outcomes, total trial cost, academic impact and use of trial results. The survey had 80 complete sets of responses (51% response rate). A trial designed to address an Unmet Need was most commonly ranked as the most important with a weight of 24.4%, followed by trials demonstrating improved Patient Outcomes with a weight of 21.2%. The value calculated for each trial allowed for their clear delineation and thus a final value ranking for each of the six trials. ConclusionWe confirmed that the use of the decision tool for valuing clinical trials is feasible and that the results are face valid based on the evaluation of six trials. A proof-of-concept applying this tool to a larger set of trials with an external validation is currently underway.
Zavala Wong, G.; Rosenauer, S.; Church, C.; Sherifali, D.; Racey, M.; Grider, K.; Moreno, A. N.; Lagrone, L. N.; The 2025 Design for Implementation (DFI) Authorship Group,
Show abstract
IntroductionRepresentatives of the trauma community have voiced a need for a new approach to developing clinical guidance. In this study, we test the initial acceptability of a proposed 12-step approach that aims to reduce the current clinical guidance timeline from more than 24 months to 24 weeks. MethodsInvestigators hypothesized that artificial intelligence (AI) tools could be leveraged to improve and make the process of clinical guidance development more efficient, facilitating AI initial output that could later be reviewed by subject matter experts (SMEs). Ensuring ethical standards and a collaborative design. Following the agile methodology, emphasizing continuous delivery and improvement, and the Practical, Robust Implementation and Sustainability Model (PRISM) framework, the investigators drafted a 12-step approach to clinical guidance development in 24 weeks. The process starts with the selection of a clinical topic and culminates in a bedside-ready clinical decision tree. ResultsThe 2025 Design for Implementation: The Future of Trauma Research & Clinical Guidance conference participants were invited to reflect on this new 12-step approach during two breakout sessions. Participants included a broad range of trauma providers, methodologists, patient representatives, technology, and marketing experts. Their recommendations highlighted: 1) multidisciplinary involvement, 2) need for resource-stratified recommendations, and 3) user-friendly features (offline and multilingual access). On a post conference survey (n=56), 64% were confident in AI accelerating the current development process. ConclusionsThe current landscape of clinical guidance offers significant opportunities for improvement. Key areas for enhancement include promoting collaboration across multiple disciplines and organizations, developing recommendations that consider resource variations, and utilizing new technologies, such as AI, to expedite the development process. This is crucial because ongoing delays lead to practices lagging behind current evidence. Further research is needed to rigorously test and refine how responsible use of AI can be integrated into expediting evidence integration into clinical guidance. Key MessagesO_ST_ABSWhat is already known on this topicC_ST_ABSCurrent clinical guidance typically takes 1-2 years to develop. Moreover, clinical guidance may not be published until a year or more after its completion, long after some recommendations become outdated, contributing to lagged evidence-informed practice. What this study addsThis study shares and tests the initial acceptability of a novel approach that aims to reduce the current clinical guidance timeline from 24 months to 24 weeks. It leverages existing artificial intelligence tools but with the critical input of subject matter experts (SMEs), ensuring ethical standards and collaborative design. SMEs shed light on critical steps and key areas that future clinical guidance needs to consider. How this study might affect research, practice or policyThe current landscape of clinical guidance offers significant opportunities for improvement. Key areas for enhancement include promoting collaboration across multiple disciplines and organizations, developing recommendations that consider resource variations, and utilizing new technologies, such as artificial intelligence, to expedite the development process.
Serra-Burriel, M.; Martin-Bassols, N.; Perenyi, G.; Vokinger, K. N.
Show abstract
This study analyzed the effects of patent expiration on drug prices in eight countries (US, UK, Cana-da, Australia, Japan, France, Germany, and Switzerland) and the impact of these price dynamics in cost-effectiveness assessments. First, using an event study design, we showed that average prices of drugs substantially decreased eight years after patent expiration. Then, to assess the implications of this finding for cost-effectiveness assessments, a theoretical cost-effectiveness model simulated two real-world scenarios: (1) the comparator drug was a generic and the patent of the new drug expired after market entry; (2) the comparator drug was also under patent protection, but the patent expired prior to the patent of the new drug. Not accounting for genericization or patent expiration of the com-parator drug resulted in an underestimation or overestimation of the incremental cost-effectiveness ratios, respectively. Our pricing dynamic estimates can be applied to base-case analyses of cost-effectiveness models.
Mitchell, H.; Xin, Q.; Hide, J.; Halin, C.; Sunil Tadwalkar, S.; Hashim, S.; Hudson, R.
Show abstract
BackgroundNational Institute for Health and Care Excellence (NICE) data regarding manufacturer-driven terminations indicate that some patients in the United Kingdom (UK) are unable to access treatments that are available in other European countries, which may result in reduced survival and quality of life (QoL). This study aims to quantify the health impact of NICE appraisals for multi-indication products terminated for reasons not related to clinical trial failure on the UK population. MethodsTerminated NICE appraisals (2014-2023) for multi-indication products were identified and a targeted literature search was conducted to identify data on the health impact of the interventions. The potential incremental quality-adjusted life year (QALY) loss and impact on overall survival (OS), progression-free survival (PFS), and QoL was calculated. ResultsOver 16,000 QALYs/year were potentially lost (with one QALY equal to one year of life in perfect health) across approximately 829,000 patients in the UK due to NICE appraisals for multi-indication products being terminated for reasons not related to clinical trial failure. Across oncology indications (approximately 18,900 patients), OS and PFS may have been reduced by over 9,400 years and 9,000 years, respectively. The potential impact of the treatments for non-oncology indications for which NICE appraisals were terminated on QoL was an incremental improvement of 13% (weighted average). ConclusionsDue to the increasing number of NICE terminations for multi-indication products, patients cannot access therapies that could lengthen their lives and increase their QoL. As the UK uniform pricing policy is likely to influence manufacturer-driven terminations, introducing alternative reimbursement arrangements such as indication-based pricing (IBP) agreements to ensure that prices remain commensurate with therapeutic value could improve access to therapies in the UK, thereby improving public health. HighlightsO_LINational Institute for Health and Care Excellence (NICE) termination data indicate that some patients in the United Kingdom (UK) are unable to access treatments available in other European countries, which could potentially prolong their lives and improve their quality of life (QoL) C_LIO_LIAcross approximately 829,000 patients in the UK, over 16,000 quality-adjusted life years (QALYs) per year (with one QALY equal to one year of life in perfect health) are potentially lost as a result of NICE appraisals for multi-indication products that have been terminated for reasons not related to clinical trial failure C_LIO_LIAssessing reimbursement options such as indication-based pricing (IBP) agreements for treatments that would typically not meet NICEs cost-effectiveness criteria at the current price provides an opportunity to improve access to therapies in the UK, thereby improving public health C_LI
Pinsent, A.; Weston, G.; Adams, E.; Linley, W.; Hawkins, N.; Schwenkglenks, M.; Williams, C. H.; Toward, T.
Show abstract
ObjectivesDravet syndrome (DS) is a rare, lifelong epileptic encephalopathy characterised by frequent and severe seizures associated with premature mortality. Typically diagnosed in infancy, patients also experience progressive behavioural, motor-function and cognitive decline. Twenty percent of patients do not reach adulthood. Quality of life (QoL) is impaired for both patients and their carers. Reducing convulsive seizure frequency, increasing seizure free days (SFDs) and improving patient/carer QoL are primary treatment goals in DS. This study explored the relationship between SFDs and patients and carers QoL to inform a cost-utility analysis of fenfluramine. MethodsIn fenfluramine registration studies, patients (or their carer proxies) completed the Paediatric Quality of Life inventory (PedsQOL). These data were mapped to EuroQol-5 Dimensions Youth version (EQ-5D-Y) to provide patient utilities. Carer utilities were collected using EQ-5D-5L and mapped to EQ-5D-3L to align patient and carer QoL on the same scale. Linear mixed-effects and panel regression models were tested and Hausman tests identified the most appropriate approach for each group. On this basis, a linear mixed-effects regression model was used to examine the relationships between patient EQ-5D-Y, and clinically relevant variables (age, frequency of SFDs per 28-days, motor impairments and treatment dose). A linear panel regression model examined the relationship between SFDs and carer QoL. ResultsAdjusting for age and underlying comorbidities, the patient regression model showed that SFDs per 28-days was a significant predictor of QoL. Each additional patient-SFD increased utility by 0.005 (p<.001). The carer linear panel model also showed that increasing SFDs per 28-days was a significant predictor of improved QoL. Each additional SFD increased carer utility by 0.014 (p<.001). SignificanceThis regression framework highlights that SFDs are significantly correlated with both patients and carers QoL. Treatment with effective antiseizure medications that increase SFDs, directly improves QoL for patients and their carers. Short summaryDS patients experience daily severe seizures with progressive deterioration in their physical, cognitive and behavioural development ("comorbidities"), which substantially impacts the QoL of patients and their carers. Reducing seizure frequency and increasing Seizure Free Days (SFDs) are key treatment goals. This study examined the relationship between seizures and patients and carers QoL. Regression analyses were conducted using data from the fenfluramine registration studies and confirmed increasing SFDs directly, and quantifiably, improved patient and carer QoL (utilities per additional SFD per 28-days: patient=0.005 and carer=0.014). These analyses demonstrate that effective antiseizure treatment can directly and profoundly improve patients and carers QoL.
Bowen, H. P.; O'Loughlin, G.; Drake, C.; Schleicher, C.; Schulthess, D.
Show abstract
BackgroundThe Most Favored Nation (MFN) policy is a mechanism that incorporates foreign prices to determine the maximum allowable net price for any branded drug within US government-funded healthcare. Two proposed rules, the Global Benchmark for Efficient Drug Pricing ("GLOBE") (90 Fed. Reg. 60,244) for Medicare Part B and the Guarding US Medicare Against Rising Drug Costs ("GUARD") (90 Fed. Reg. 60,338) for Medicare Part D, invoke the Center for Medicare and Medicaid Innovation Centers payment and service model demonstration and waiver authority, under Section 1115A of the Social Security Act (42 U.S.C. [§] 1315a), to calculate the US MFN price which is the lowest average price within a basket of specified foreign countries. Unlike voluntary manufacturer agreements, GLOBE and GUARD would mandate participation from all applicable manufacturers. MethodsWe derive MFNs potential impact on Medicare pricing from a proprietary dataset provided by IQVIA which contained net prices for the top 37 oncology products by total US sales from January 1, 2019 through June 30, 2025 ranked by total US sales in the following countries: Australia, Belgium, France, Germany, Ireland, Italy, South Africa, Spain, Switzerland, the UK, and the US. For each drug, we select the lowest GDP-adjusted international price from a basket of those countries within 60% of the US GDP per capita, adjusted for purchasing power parity, and calculate the reduction in US price required to match its MFN price, and hence the corresponding reduction in revenues under MFN. A retrospective Net Present Value (NPV) analysis is then used to address the counterfactual question of whether each drug would have been developed had MFN pricing been in place at the time of its FDA approval. ResultsUnder MFN, the average reduction in US prices across our drug cohort was 67%. Eighty-four percent of the 37 cancer drugs in our cohort evidenced a negative NPV if MFN had been in place at the time of their FDA approval and the commercial market is impacted. When the analysis is restricted to MFNs impact on Medicare, the indications for these lost drugs have a total US population of 2.4 million patients. When the analysis is combined across the Medicare and commercial markets, the loss of lead indications impacts over 15 million US patients. ConclusionsMandatory MFN policies reduce the financial incentives required to develop cancer medicines; our projections show a substantial decline in new cancer drug launches and will likely lead companies to pursue indications for populations outside Medicares authority. If so, MFN will reduce the number of new therapies for the very population the Executive Orders are allegedly designed to aid: the Medicare-aged population who require effective new therapies in areas of high unmet medical need, such as late-stage cancers. This creates the perverse outcome of a policy nominally designed to help Medicare beneficiaries by instead redirecting innovation away from their most urgent therapeutic needs.
Creeden, J.; Olivecrona, M.; Soriano, A.
Show abstract
Background. Tertiary clinical genomics reports condense layered molecular findings into documents that treating oncologists must read, translate, and act upon; manual summarisation of these reports is time-consuming and variable. Tools that assist summarisation and translation into local languages are emerging, yet the field lacks an agreed methodology for evaluating such tools before any downstream clinical use. The appropriate first endpoint is fidelity of the generated summary to its source report, assessed by qualified human raters under blinded scoring, not downstream variant classification. Methods. QNOMX-VHIR-CPSP-001 Phase 1 is a single-site, non-interventional clinical performance study conducted at Vall d'Hebron Institut de Recerca (VHIR) under ISO 20916:2019 as a Clinical Performance Study Protocol. De-identified tertiary cancer genomics reports from pediatric oncology cases are summarised by the AI-assisted summarisation system under evaluation and, in parallel, by the standard manual workflow. Qualified raters score both summary types against the source genomics report using the Quality Summary Index (QSI), a six-dimension, five-point rubric adapted from the Provider Documentation Summarization Quality Instrument, under a blinded, counterbalanced, two-period crossover with a minimum fourteen-day washout. Two co-primary composite endpoints, content and presentation, are analysed for non-inferiority under a Bayesian hierarchical model, with a frequentist linear mixed model as the convergence check. Inter-rater reliability is reported as Krippendorff's ; a Monte-Carlo power analysis of the fixed clustered design is pre-specified. Discussion. The design isolates summarisation quality from clinical decision-making by scoring both summary types against the same source report under blinding, counterbalancing, and a fourteen-day washout. Conclusion. The QSI rubric, the counterbalanced crossover, and the pre-specified Bayesian primary with frequentist convergence check define a replicable protocol for early-stage evaluation of AI-assisted summarisation in tertiary genomics reporting; observed variance components will inform sample-size determination for Phase 2.
Chechulina, V.; Chu, A.; Lam, Q. H.; Kabir-Bahk, C.; Ali, S.
Show abstract
ObjectivesHealth system resources are limited, and decision makers often need to make trade-offs between equity and efficiency. Such trade-offs are guided by public values that are elicited using choice experiments. There is no standard approach to elicit equity-efficiency trade-offs. Previous studies have found significant variability in public values, raising concerns about the robustness of trade-off experiments. The objective of this systematic review was to determine whether and how equity-efficiency trade-off studies consider validity, reliability and framing effects. MethodsWe searched Medline, EMBASE, and Web of Science in January 2024 for health-related equity-efficiency trade-off studies. Two reviewers independently screened titles and abstracts, followed by full-text review, data extraction, and quality assessment. Results115 equity-efficiency trade-off studies were identified, of which 29 studies (25.2%) investigated validity, reliability, and/or framing effects. 14 (12.2%) studies assessed validity, 9 (7.8%) studies assessed reliability, and 10 (8.7%) assessed framing or cognitive effects. Validity was most frequently assessed by comparing results to hypothesized expectations, while reliability was commonly assessed by providing a repeated test or questionnaire. Framing and cognitive effects were assessed by varying question order or changing the wording or framing of the scenario. 19 of 23 studies reported high or acceptable validity or reliability, and 5 of 10 studies found no significant framing or cognitive effects. ConclusionThis systematic review highlights the need to consider robustness of elicited values in choice experiments. It will also guide future choice experiments that aim to estimate inequity aversion.
Doungsong, K.; Hartfiel, N.; Gladman, J.; Harwood, R. H.; Edwards, R. T.
Show abstract
BackgroundRegular exercise and community engagement may slow the rate of function loss for people with dementia. However, the evidence is uncertain regarding the cost-effectiveness and social return on investment (SROI) of home exercise with community referral for people with dementia. This study aimed to compare the social value generated from the in-person PrAISED programme delivered before March 2020 with a blended PrAISED programme delivered after March 2020. MethodsSROI analysis was conducted alongside a randomised controlled trial (RCT). Of 205 patient participants and their carers who completed cost data, 61 completed an in-person programme before March 2020. Due to COVID-19 pandemic restrictions, 144 patient participants completed a blended programme consisting of a combination of in-person visits, phone calls and video conferencing with multidisciplinary team (MDT) members. SROI analysis compared in-person and blended delivery formats. Five relevant and material outcomes were identified: three outcomes for patient participants (fear of falling, health-related quality of life, and social connection); one outcome for carer participants (carer strain index), and one outcome for the NHS (health service resource use). Data were collected at baseline and a 12-month follow-up. ResultsThe in-person PrAISED programme generated SROI ratios ranging from {pound}0.58 to {pound}2.33 for every {pound}1 invested. In-person PrAISED patient participants gained social value from improved health-related quality of life, social connection, and less fear of falling. In-person PrAISED carer participants acquired social value from less carer strain. The NHS gained benefit from less health care service resource use. However, the blended PrAISED programme generated lower SROI ratios ranging from a negative ratio to {pound}0.08: {pound}1. ConclusionCompared with the blended programme, the PrAISED in-person programme generated higher SROI ratios for people with early dementia. During the COVID-19 pandemic and its restrictions, a blended delivery of the programme and the curtailment of community activities resulted in lower SROI ratios during this period. An in-person PrAISED intervention with community referral is likely to provide better value for money than a blended one with limited community referral, despite the greater costs of the former. Trial registrationISRCTN15320670
Ezeofor, V. S.; Hartfiel, N.; Doungsong, K.; Goldberg, S.; van der Wardt, V.; Howe, L.; Gladman, J.; Harwood, R. H.; Edwards, R. T.
Show abstract
BackgroundThe effectiveness of exercise interventions to improve activities of daily living function in people with dementia is inconclusive. This study aimed to assess the long-term cost-effectiveness of the PrAISED intervention from a National Health Service (NHS) perspective. MethodThis novel robust economic analysis used a Markov model to evaluate the incremental cost-effectiveness ratio (ICER) over a lifetime horizon of 15 years for a cohort of patients. Sensitivity analyses were conducted to investigate the uncertainty and robustness of high-impacting parameters and results. ResultsThis study included 365 adults, aged 65 years and above with 183 and 182 randomised to the PrAISED and standard care groups respectively. The PrAISED intervention had mean per-patient cost of {pound}60,465 for the PrAISED arm and {pound}54,604 for standard care. The Praised intervention gained an incremental QALYs of 0.05 resulting in an ICER of {pound}129,614 per QALY. The sensitivity analysis of the intervention cost varied the ICER value between {pound}68,173 and {pound}191,054/QALY. To achieve the recommended NICE willingness to pay threshold value of less than {pound}30,000/QALYs would require the intervention cost to be reduced from {pound}1,236 (current cost) to {pound}263 to break even and be cost-effective. The sensitivity analyses revealed that there was a 40% probability of standard care dominating the PrAISED treatment. ConclusionsAlthough the PrAISED intervention was a low-cost intervention, it did not produce a cost-effective intervention in this analysis. The flexibility of the PrAISED program to adapt to government policy during the COVID-19 pandemic was positive. Trial registrationISRCTN15320670
Khan, D. Z.; Adams, T.; Wijekoon, A.; Ramirez Herrera, R.; Bano, S.; McCulloch, P.; Stoyanov, D.; Clarkson, M. J.; Costanza, E.; Blandford, A.; Marcus, H.; CARES Evaluation Group,
Show abstract
Artificial intelligence (AI) decision support systems for surgery hold promise but face barriers to adoption, particularly around the interpretability of their outputs. We conducted an international cross-sectional survey of 47 neurosurgeons to evaluate perspectives on literature-derived explanation techniques for AI-generated anatomical segmentations, using endoscopic pituitary surgery as a high-risk exemplar. Participants ranked certainty scores, certainty maps, saliency maps, scene similarity scores, and nearest-neighbour illustrations, and rated them using a modified Explanation Satisfaction Scale alongside free-text feedback. Certainty-based techniques were consistently ranked and rated highest for interpretability - valued for aligning with surgical decision-making by conveying confidence (via scores) and anatomical boundaries (via maps). Saliency- and similarity-based methods were judged less clinically relevant and better suited to educational settings. Certainty-based explanations, therefore, appear most acceptable to surgeons for clinical integration of decision support systems, though their impact on AI acceptability, trust calibration, and performance requires prospective evaluation across surgical domains.
Yuan, F.; Spence, J. D.; Tarride, J.-E.
Show abstract
BACKGROUNDAmong patients with type 2 diabetes and a history of strokes or transient ischemic attacks, pioglitazone significantly reduces the risk of recurrent stroke. The Insulin Resistance Intervention in Stroke (IRIS) trial found that pioglitazone also reduced the risks of stroke or transient ischemic attacks and new-onset diabetes among patients with insulin resistance. As reported by our previous work, the low-dose pioglitazone was found to provide most of the clinical benefit of high-dose pioglitazone, with fewer adverse effects. We report a model-based economic evaluation to determine the cost-effectiveness of the low-dose pioglitazone versus placebo. METHODSA lifetime Markov model, with an annual cycle length and five health states (event-free, myocardial infarction, stroke, new-onset diabetes, death), was developed. Transition probabilities were extracted from a subgroup of IRIS patients with insulin resistance, defined by a glycosylated hemoglobin level of 5.7% to 6.4% (mean follow-up of 5 years). Health state costs and utilities were based on public sources and literature data, respectively. Utilities were weighted by time spent in health states to calculate quality-adjusted life years (QALYs). The incremental cost per QALY gained was estimated for the population. Annual discount rates of 0%, 1.5%, and 3% were applied. In addition to deterministic analyses, probabilistic sensitivity analyses were conducted to deal with parameter uncertainty. The analyses were conducted from a Canadian public payer perspective in 2023 Canadian dollars. RESULTSThe base case results indicated that over a lifetime, the expected costs were CAN$31,534 for low-dose pioglitazone and CAN$55,076 for placebo, resulting in a cost saving of CAN$23,542 in favor of the low-dose pioglitazone. Expected QALYs were 25.10 for the low-dose pioglitazone daily and 19.32 for placebo, resulting in a difference of 5.78 QALYs in favor of low-dose pioglitazone. Probability sensitivity analyses with varying discount rates confirmed these results. CONCLUSIONSCompared with placebo, low-dose pioglitazone is the dominant strategy. Key PointsO_ST_ABSQuestionC_ST_ABSCompared with a placebo, is low-dose pioglitazone cost-effective in treating stroke/ transient ischemic attack (TIA) and new-onset diabetes in a simulated population with prediabetics? FindingsOver a lifetime, the expected costs were CAN$31,534 for low-dose pioglitazone and CAN$55,076 for placebo, resulting in a cost saving of CAN$23,542 in favor of low-dose pioglitazone. Expected QALYs were 25.10 for low-dose pioglitazone daily and 19.32 for placebo, resulting in a difference of 5.78 QALYs in favor of low-dose pioglitazone. Sensitivity and scenario analyses confirmed the results. MeaningThe model-based economic evaluation indicates that low-dose pioglitazone, compared with placebo, is the dominant strategy.
Powers, S.; Anderson, K.; Tan, W.-H.; Gwaltney, A.; Potter, S.; Tillmann, J. E.; Daniel, M.; Thurm, A.; Farmer, C.; Clinch, S.; Squassante, L.; Tjeertes, J.; Vincenzi, B.; Buzasi, K.; Wheeler, A. C.; Sadhwani, A.
Show abstract
ObjectivesTo develop within-patient meaningful score differences (MSDs) on the Bayley Scales of Infant Development, Fourth Edition (Bayley-4), and the Vineland Adaptive Behavior Scales, Third Edition (Vineland-3), for individuals with Angelman syndrome (AS). MethodsA Delphi method, involving a panel of 19 caregivers of individuals with AS, was used to establish MSDs for Bayley-4 and Vineland-3 Growth Scale Values. MSD was defined as the smallest change that would noticeably impact the daily functioning of an individual with AS or family quality of life in a way that was important to the individual with AS or their family. For each subscale of the Bayley-4 and Vineland-3, the panel was presented with 2 to 4 vignettes describing varying levels of baseline functioning and asked to select a MSD from a range of potential values. An iterative process involving three rounds of ratings and two rounds of discussion was used to build consensus. The median caregiver rating from round 3 was retained as the final recommended MSD value for each vignette. ResultsFinal MSD ratings for the five subscales of Bayley-4 and 10 subscales of the Vineland-3 had an agreement rate of 70% or higher. MSD thresholds for each subscale were not single cut-offs, but rather reflected a range of MSD values dependent on level of baseline functioning. ConclusionsThe Delphi Panel method incorporates the caregiver perspective to provide preliminary estimates of what constitutes meaningful within person change on the Bayley-4 and Vineland-3 in individuals with AS with various levels of baseline functioning. Highlights To acquire regulatory approval in drug development, sponsors must demonstrate both statistical significance and clinical meaningfulness of a treatment effect.While several clinical trials are underway in AS, within person meaningful score difference thresholds are not yet established for the most commonly used outcome measures, namely the Bayley and Vineland. Aligning with FDA guidance, we have developed an innovative qualitative approach using a Delphi panel to incorporate caregiver perspectives in defining meaningful change and generated preliminary patient-informed meaningful score differences (MSDs) for individuals with Angelman Syndrome. What caregivers of individuals with AS consider to be a MSD on the measures depends primarily on the baseline severity of their childs presentation.